As is known in the art, there exists a class of circuits and systems which emulate higher order brain functions such as memory, learning and or pattern perception/recognition. Such systems are generally referred to as neural networks or adaptive classifiers. Neural networks may be trained to model regions in which particular features or characteristics of an input signal are distributed. By having an accurate model of such regions, a neural network may thus recognize if unknown data fed thereto belongs to a particular class represented by the modelled region.
Such modeling is accomplished by presenting to the neural network a number of training signals belonging to known classes of interest. Each of the training signals is provided having a predetermined number of preselected characteristics or features. During a training mode, each of the features and the class to which a signal having such features belongs are provided to the neural network. The neural network stores the information and thus generates a model of the region which includes signals of a particular class.
Typically however, the neural network includes a plurality of modeled regions which represent a corresponding plurality of different classes. In many practical problems such modeled regions have overlapping portions. When an unknown input signal has characteristics which map into a region where no overlap exists among the portions the neural network is able to classify the unknown input signal into one of the classes.
However, when the unknown signal has characteristics which map the signal into a region where portions of the different classes overlap, the neural network is unable to classify the unknown input signal with certainty, because the region belongs to more than one class. Thus, the network provides acceptable results when class regions are separable, that is, when no overlap exists between the class regions.
One approach to solving this problem caused by overlapping class regions is by using a so-called Probabilistic Neural Network (PNN). The PNN approach Computes probability estimates by requiring each pattern in the training set to be stored in a first layer of weights.
One problem with this approach however, is that in some applications a large number of training patterns must be used. In many practical neural network applications, the number of training patterns is an exponential function of the number of input features. Thus, practical problems generally require a prohibitively large number of training patterns which make implementation of a probabilistic neural network impractical.
Due to the large number of training patterns required, a relatively large amount of memory is required. In some applications the amount of available memory may be limited. Furthermore, memory size is especially important when it is desirable to manufacture the neural network as an integrated circuit.
Moreover, in some applications such as large-scale pattern classification tasks, it may be desirable to provide a neural network which may operate on a real-time basis. To achieve real time operation in such applications however, the neural network must be capable of processing information in a relatively rapid manner. Thus, to meet such processing speed requirements it may be desirable to provide a neural network as a semiconductor integrated circuit.
However, due to the complexity of the resultant circuits associated with the use of predecessor neural network architecture, it has been relatively difficult to construct a neural network integrated circuit which was useful for typical classification problems. Neural networks able to operate in a wide variety of processing applications generally have neural network architectures requiring a relatively large number of processing element inter-connectivity. This results in a reduction of the number of processing elements which may be practically implemented in an integrated circuit using present manufacturing techniques.
It would thus be desirable to provide an adaptive classification system as an integrated circuit device which is capable of resolving ambiguities which occur when class distributions overlap and prototypes from more than one class are likely to fire at the same time in response to a single unknown input pattern.